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(54) Method for distributed denial-of-service attack mitigation by selective black-holing in MPLS 
VPNs 



(57) A system and method for aiding the handling of 
DDoS attacks in which VPN traffic entering an ISP net- 
work (10) at some points will be black-holed (12), while 
VPN traffic entering the ISP network at other points will 
be routed, as it should be, to the system-under-attack 



(14). Thus, the system-under-attack (14) is made avail- 
able to some of the user community and made unavail- 
able to suspect portions of the user community. Further- 
more, the number of entry points where black-holing of 
VPN traffic occurs can be selected and changed in real- 
time during a DDoS attack. 



<Fig. 4 



16 

RR1 * 



R1 




it MP-iE 

,37 INJECTED \\ /' ^—^T™ 



CO 
CO 

to 



Q. 

LU 



10.10.10, 
WITH A HIGHER 
PREFERENCE AND 
COMMUNITY VALUE 
OF RR3 




EP 1 566 947 A1 



Description 

BACKGROUND OF THE INVENTION 

Fietd of the Invention 

[0001] The present invention relates generally to Vir- 
tual Private Networks (VPN) based on Multiprotocol La- 
bel Switching (MPLS), and more particularly to redirect- 
ing or rerouting VPN traffic in response to an attack 
caused by an attacker flooding a victim's host system 
with one or more ol several types of attack traffic. 

Background of the Invention 

[0002] In a Distributed Denial of Service (DDoS) at* 
tack, an attacker takes control of one or more hosts 
(daemons) and uses the daemons to send an enormous 
amount of traffic to a, for example, web site so that no 
other traffic can get through to the website. In essence, 
the website is clogged or jammed with traffic. The fre- 
quency of DDoS attacks in the Internet has grown in the 
past several years. The flooding of a victim's host sys- 
tem with attack traffic causes legitimate users of the vic- 
tim's host system to be denied access to applications 
running on the System-Under-Attack (SUA). The appli- 
cation can be a web server, a file server, a Domain Name 
System (DNS) server, or other Internet related service 
or device. The legitimate users cannot access the ap- 
plication due to Central Processing Unit (CPU) and/or 
bandwidth exhaustion on the system under attack. An 
attack may have a distributed nature due to the attack 
traffic being from random, usually spoofed, source IP 
addresses and originating from many daemon hosts. Al- 
so, the attack traffic may enter the victim's Virtual Private 
Network (VPN) network from various entry points. 
[0003] It is well known that DDoS attacks are among 
the most difficult types of attacks to defend against. A 
system is vulnerable to a DDoS attack simply by being 
connected to the internet. The federal government is in- 
creasingly aware of DDoS attacks and may propose that 
federal agencies only utilize ISPs that have DDoS pro- 
tection in their networks. 

[0004] For a DDoS attack to be successful in a Multi- 
protocol Label Switched (MPLS) Virtual Private Network 
(VPN) environment, the master, daemons and the sys- 
tem-under-attack (the victim) have to belong to the 
same or mutually accessible VPNs. In many other re- 
spects, a DDoS attack in a MPLS VPN is similar to a 
DDoS attack in an IP network. 

BRIEF SUMMARY OF THE INVENTION 

[0005] A more complete appreciation of the present 
invention and the scope thereof can be obtained from 
the accompanying drawings, which are briefly summa- 
rized below, the following detailed description of the 
presently-preferred embodiments of the invention, and 



2 

the appended claims. 

[0006] An embodiment in accordance with the 
present invention is an ISP network that includes a plu- 
rality of edge routers. A plurality of core routers is found 

5 within the ISP network and is adapted to allow commu- 
nication between the plurality of edge routers. A VPN 
application, such as a website or database, is hooked 
up to be in communication with one of the edge routers. 
The VPN application has an IP address. There also ex- 

10 ists a black-hole router that is in communication with ei- 
ther an edge router or one of the core routers. The black- 
hole router is adapted to black-hole at least some of the 
traffic having the IP address of the VPN application. The 
black-hole router can further selectively black-hole traf- 

f* fic addressed to the VPN application that is routed 
through one or more of the edge routers into the ISP 
network, 

[0007] In the embodiment of the present invention, the 
black-hole router is adapted to inject a dummy or bogus 
20 ip address into the ISP network. The bogus IP address 
is the same address as the VPN application's address 
but has a higher preference value and a community val- 
ue that selects a number of the edge routers and re- 
quires them to address VPN traffic, having the VPN ap- 
25 plication's IP address, to the bogus address thereby 
black-holing the traffic. Embodiments of the present in- 
vention are well suited for handling DDoS attacks on a 
website such that some of the traffic is diverted from get- 
ting to the website-under-attack and other VPN traffic is 
30 allowed to continue to the website-under-attack. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0008] The foregoing and other advantages of the in- 
35 vention will become more apparent upon reading the fol- 
lowing Detailed Description of Embodiments of the In- 
vention in conjunction and reference with the drawings. 

FIGURE 1 depicts a general ISP network operating 
40 in part as a MPLS VPN using a known total black- 
holing technique; 

FIGURE 2 is a graph comparing traffic profiles that 
may be utilized with an embodiment of the present 
invention; 

45 FIGURE 3 depicts a general ISP network operating 
in part as a MPLS VPN using a selective black-hol- 
ing technique in accordance with an embodiment of 
the present invention; and 
FIGURE 4 depicts exemplary black-holing in a 
so MPLS VPN using MP-iBGP route filtering. 

[0009] While the invention is susceptible to various 
modifications and alternative forms, (for example, the 
invention can be easily adapted to a case with multiple 
ss black-hole routers.) specific embodiments have been 
shown by way of example in the drawings and will be 
described in detail herein. It should be understood, how- 
ever, that the invention is not intended to be limited to 
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the particular forms disclosed. Rather, the invention is 
to cover all reasonable modifications, equivalents, and 
alternatives falling within the spirit and scope of the in- 
vention as defined by the appended claims. 

DETAILED DESCRIPTION OF THE INVENTION 

[0010] The present invention will now be described 
more fully hereinafter with reference to the accompany- 
ing drawings in which preferred embodiments of the in- 
vention are shown. This invention may, however, be em- 
bodied in many different forms and should not be con- 
structed as limited to the embodiments set forth herein; 
rather, these embodiments are provided so that this dis- 
closure will be thorough and complete, and will fully con- 
vey the scope of the invention to those skilled in the art. 
[0011] Referring to FIGURE 1, currently the prevalent 
black-holing defense against a DDoS attack consists of 
redirecting all of the traffic destined to the system under 
attack to another router (or host) called a black-hole 
router (or host) 12. The redirected attack traffic is then 
analyzed to determine the origin of the attack or, the re- 
directed traffic is simply dropped into a "black-hole" 
(hence the term black-holing). In either case, all of the 
traffic addressed for the system-under-attack 14 is redi- 
rected to the black-hole router 12. The system-under- 
attack 14 becomes unavailable to all users, legitimate 
or attacking, for the duration of the black-holing. Total 
(non-selective) black-holing is therefore not an effective 
defense against a DDoS attack since the attacker's in- 
tention of shutting down a VPN application or creating 
a den ial-of -service to the system-under-attack is not 
averted. 

[0012] FIGURE 1 further depicts edge routers R1 
through R6. Edge routers are routers on the edge of a 
VPN or the routers of an ISP that are in direct commu- 
nication with customers. There is a plurality of other rout- 
ers within or internal to the ISP network 1 0 that are not 
specifically shown called core routers. The routers in- 
ternal to the ISP network that are part of the VPN are 
basically transparent routers that are used for backbone 
routing of traffic throughout the VPN. One of the funda- 
mental differences between a MPLS VPN ISP and a reg- 
ular ISP network is that each intermediate router of a 
regular ISP network has to determine the BGP next-hop 
router for every IP address known through BGP. Con- 
versely, in a MPLS VPN ISP network the internal, inter- 
mediate routers are not concerned with BGP next-hop 
routers, but instead are preconfigured to pass traffic 
from any edge router R1-R6 to any other edge router 
R1 -R6 through the use of MPLS. 
[0013] Referring to FIGURE 1 , one of the exemplary 
techniques for mitigating an attack is for an MPLS VPN 
ISP network in accordance with the present invention to 
introduce a bogus route for traffic to take to the website. 
A black-hole router 12 is introduced to the MPLS VPN 
ISP network 1 0. The black-hole router 1 2 contains a bo- 
gus address to the website-under-attack 14. The bogus 



address and the website under attack each have the 
same address except the bogus address has a higher 
priority or preference. The higher preference requires 
that aj] traffic going to the website-under-attack's ad- 
5 dress will be diverted or black-holed to the black-hole 
router 12. The traffic flow arrows 16 indicate the direc- 
tion of all traffic from the edge routers R1 -R6 addressed 
to the website-under-attack 14 that is redirected to the 
black-hole router 12 via the bogus address having a 
10 higher priority than the website address. The bogus ad- 
dress is the same address as the address for the web- 
site-under-attack 14, except the bogus address has a 
higher priority or preference. 

[0014] Since all the traffic intended for the website- 
's under-attack is redirected to the black-hole router 12, 
then even the valid, non-attacking, traffic is black-holed. 
All traffic intended for the website-under-attack is divert- 
ed. The attacker has essentially won the attack and shut 
down the website-under-attack 14 while the MPLS VPN 
20 ISP and their customer(s) figure out what to do. 

[0015] The MPLS VPN ISP analyzes the redirected 
traffic being received at the black-hole router 12 in order 
to trace where the attack traffic is coming from so that 
a better defense to the attack can be established. Mean- 
25 while, legitimate users of the system are denied access 
to the website-under-attack 1 4 because legitimate traffic 
is also being black-holed. 

[0016] There are a couple of popular types of attack 
traffic used by attackers. For example, SYN attack traffic 
30 j S used as part of a message sent using Transmission 
Control Protocol (TCP). In TCP, a packet of information 
that initiates a TCP connection is called a SYN packet. 
An attacker sends many, many SYN packets to the web- 
site-under-attack. In turn, the website-under-attack, re- 
35 sponds by attempting to create a TCP connection to all 
the incoming SYN packets. The proper response to a 
SYN packet by a website is to reply to each SYN packet 
with another SYN packet and then wait for another type 
of packet called an ACK packet to acknowledge receipt 
*o of the website's SYN packet and complete the "TCP 
Handshake" establishing a TCP connection between 
the website and the user. 

[001 7] In a SYN attack, the ACK is never sent back to 
the website-under-attack. Thus, the website-under-at- 
45 tack is bombarded with SYN packets. The website-un- 
der-attack replies to each SYN packet with appropriate 
SYN packets and waits for an ACK for each reply, but 
never gets any. The website-under-attack receives mil- 
lions of SYN packets, but never receives an ACK. The 
50 website-under-attack is trying to set up all the potential 
TCP connections, but the TCP handshakes are never 
completed. The website-under-attack's bandwidth and 
processor is consumed by servicing the incomplete TCP 
handshakes and is in effect shut down to valid traffic. 
55 [0018] In the other type of ICMP attack called a PING 
attack, the attacker utilizes the "PING" utility of the IC- 
M P. The PING utility is used to determine whether a spe- 
cific IP address is accessible. A PING packet is sent to 
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a specified address and a reply is then sent by the spec- 
ified address. In a PING attack, millions of PINGs are 
sent to the website-under-attack and the CPU becomes 
overwhelmed by attempting to respond to all the PINGs. 
In effect, the bandwidth of both the website's CPU(s) 
and communication channels is used up so that valid 
traffic to the website-under-attack is blocked. 
[0019] When all traffic to a website-under-attack is di- 
verted by an MPLS VPN ISP to a black-hole router 12, 
it is referred to as non-selective black-holing. In accord- 
ance with an embodiment of the invention, a new tech- 
nique that provides additional options for handling an 
attack on an application associated with an ISP is called 
selective-black-holing. As depicted in FIGURE 3, selec- 
tive-black-holing comprises selectively diverting traffic, 
destined to the system-under-attack 14, that enters a 
MPLS VPN ISP network 1 0 from a subset of entry nodes 
(R1 , R2, R3) in the MPLS VPN ISP network and allowing 
traffic entering the MPLS VPN ISP network 1 0 from oth- 
er entry points (R4, R5, R6) to reach the system-under- 
attack 1 4. This is achieved via dynamic routing protocols 
in combination with community-based route filtering. 
[0020] The ISP must either determine or predeter- 
mine which edge routers should black-hole potential at- 
tack traffic and which edge router can allow the potential 
attack traffic to proceed to the address of the system- 
under-attack 14. There are different schemes that can 
be used to choose the entry nodes (routers) or edge 
routers that implement black-holing of potential attack 
traffic. One approach is to selectively black-hole traffic 
consecutively at each entry point (edge router) of the 
MPLS VPN ISP network. The black-holed traffic can be 
analyzed to determine the ratio of attack traffic to legit- 
imate traffic at each entry point. Once traffic that is ad- 
dressed to the system-under-attack from each selected 
or all entry points is analyzed, black-holing of the traffic 
can be limited to the entry points that have the highest 
percent of attack traffic. 

[0021] Another technique, as shown in FIGURE 2, for 
using selective-black-holing to analyze potential origins 
of a DDoS attack is to characterize the intensity of the 
attack at the major entry points of the DDoS attack traffic 
by comparing an "average" profile of traffic to a "current 0 
profile of traffic (i.e., a profile from the past five or ten 
minute interval) at each of the entry points of the MPLS 
VPN ISP network. The average profile of traffic can be 
pre-constructed by the service provider for selected, or 
premium customers of the network. For example, the 
program utility called "NETFLOW", Cisco Corporation's 
traffic statistics collection feature, can be used to con- 
struct such profiles. 

[0022] For example, in FIGURE 2, router one R1 may 
be an edge router entry point in California. Router two 
R2 may be an edge router entry point in New York. Rout- 
er three R3 may be an edge router in Boston. Routers 
R4-R6 may be edge routers in other major cities. By 
comparing an average traffic profile with the current traf- 
fic profile for each router one may surmise that attack 



traffic mainly originating from edge routers R1 , R2 and 
R3 because the traffic addressed to the system-under- 
attack 1 4 increased significantly in the current traffic pro- 
file (i.e., during the attack) when compared to the aver- 

5 age traffic profile. 

[0023] By using selective-black-holing in accordance 
with an embodiment of the invention VPN traffic sent to 
the system under attack via routers one, two and three 
(R1 , R2 and R3) can be, black-holed to the black-hole 

10 router12shown in FIGURE 3. The traffic received at the 
black-hole router can be analyzed to determine whether 
attack traffic is present. And, if so, where the attack traf- 
fic is originating. 

[0024] Meanwhile, routers four, five, and six (R4, R5, 
is R6) can selectively be allowed to continue forwarding 
traffic, addressed to the system-under-attack, to the 
system-under-attack 1 4. Thus, the attacker has not suc- 
ceeded in "shutting down" the system-under-attack 14 
because traffic may still be received and transmitted 

20 from the system-under-attack 14. The communication 
bandwidth and/or the microprocessor(s) at the system- 
under-attack are not attempting to operate beyond their 
capabilities when majority of the attack traffic is being 
black-holed. There are various advantages to using se- 

25 lective-black-holing. For, the response time to a DDoS 
attack can be decreased since various selective-black- 
holing strategies can be predetermined and implement- 
ed as soon as a DDoS attack is discovered to be under- 
way. By using Border Gateway Protocol (BGP) routing 

30 and community-based route filtering in exemplary em- 
bodiments of the invention, a predetermined black-hol- 
ing strategy can be setup and implemented. BGP is an 
exterior gateway routing protocol that enables groups of 
routers to share routing information. BGP is commonly 

35 used within and between ISPs. It is understood that an 
exemplary network or system can utilize other dynamic 
routing protocols, besides BGP, as long as they allow 
route filtering. Once an attack has commenced, the se- 
lective-black-holing is initiated by communicating the IP 
address of the system-under-attack with a predefined 
community value and higher preference from selected 
point(s) in the ISP network. Depending on the commu- 
nity value, traffic destined to the system-under-attack 
from any number of entry points (i.e., edge routers) can 

45 be rerouted to the black-hole router. 

[0025] The black-holing strategy can also be adjusted 
in real-time during an attack. By adjusting the number 
of entry points (edge routers) that black-hole traffic that 
is addressed to the system-under-attack, the system- 

50 under-attack can be made available to part of the user- 
community during a DDoS attack. 
[0026] An embodiment of the present selective-black- 
holing technique can selectively adjust the number of 
edge routers (from none to all the routers) that direct a 

55 specific type of traffic to a black-hole router in the MPLS 
VPN ISP network. For route filtering in a selective black- 
holing MPLS VPN ISP system - iBGP (MP-iBGP) route 
filtering is utilized using community values. As stated 
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above, selective-black-holing can be implemented us- 
ing other dynamic routing protocols and filtering 
schemes besides iBGP 

[0027] Still referring to FIGU RE 3, black-holing of at- 
tack traffic is achieved by injecting, by the ISP (via any 
router including the black-hole router 12), the IP address 
of the system-under-attack 14 into a routing protocol, 
such as a BGP. The injected IP address is the same IP 
address as the address of the system-under-attack ex- 
cept that the injected I P address has a higher preference 
than the IP address of the system-under-attack. 
[0028] In another embodiment of the present inven- 
tion, route reflectors (RRs) can be implemented. Route 
reflectors are typically used in large ISPs. Referring to 
FIGURE 4, the implementation of RRs in an embodi- 
ment of the invention comprises using a community at- 
tribute in the BGP to enable selective filtering of routing 
updates in MP-iBGP sessions. Every edge routerR1-R6 
must communicate with every other edge router in the 
ISP in order to disseminate information indicating, 
among other things, the destinations that the edge rout- 
er can communicate with. In a large network dissemi- 
nating the information to all the edge routers by each 
edge router is a daunting, bandwidth consuming task. 
There can be hundreds to thousands of edge routers in 
a given ISP network. A technique for simplifying the task 
of each edge router communicating with all other edge 
routers is to use route reflectors. 
[0029] Instead of communicating with every edge 
router, each edge router informs its immediate route re- 
flector of the possible route(s) it can reach. The route 
reflector then reflects the route information, via an MP- 
iBGP mesh 22, to other route reflectors, which in turn 
communicates to their associated edge routers the rout- 
ing information. 

[0030] In FIGURE 4, all entry points (R1-R6)of an ISP 
Network 40 and the black-hole router 12 are Provider 
Edge (PE) routers in the VPN context. The PE routers 
are route reflector clients of the RRs 16, 18,20,and24. 
Selective route filtering is achieved by filtering based on 
community values in the MP-iBGP sessions. There is 
an MP-iBGP full mesh 22 among all RRs 1 6, 1 8, 20, 24. 
In this embodiment, the selectivity is limited to sets of 
RR clients, edge routers that are served by their own 
RR. That is, all of the attack and legitimate traffic enter- 
ing a set of edge routers (e.g., R1 and R2) addressed 
and destined for the system-under-attack 14 has to be 
biack-holed in the black-hole router 12 or allowed to 
reach the system-under-attack 14. 
[0031] Embodiments of the present invention can be 
adjusted or implemented as a protection for premium 
customers because average traffic profiles for each 
edge router can be pre-constructed for the premium 
customer's IP addresses. When such a customer's ap- 
plication 14 becomes the target of a DDoS attack, the 
black-hole router 12 can be preconfigured to handle an 
attack on application 14 by utilizing a set of community 
values and associated higher preferences for selective 



black-holing. 

[0032] Embodiments of the invention provide a real- 
time adjustable and selectable black-holing strategy for 
handling a DDoS attack in a manner that does not shut 
5 down the system-under-attack. Embodiments of the 
present invention and obvious variations thereof are 
contemplated as falling within the spirit and scope of the 
claimed invention, which is set forth in the following 
claims: 

10 

Claims 

1. An internet service provider (ISP) VPN network 
15 comprising: 



a plurality of edge routers; 
a plurality of core routers adapted to allow com- 
munication between said plurality of edge rout- 
ers; 

a VPN application in communication with a first 
one of said plurality of edge routers, said VPN 
application having a first IP address; and 
a black-hole router in communication with said 
core routers, said black-hole router adapted to 
inject a second IP address into said ISP VPN 
network, said second IP address comprising: 



20 



25 



30 



35 



40 



the same address as the first IP address; 
a higher preference value than said first IP 
address; and 

a community value such that when said 
second IP address is injected, a selected 
first number of edge routers direct VPN 
traffic addressed for said first IP address to 
said VPN application and a selected sec- 
ond number of edge routers direct VPN 
traffic addressed for said first IP address to 
said black hole router. 

2. The ISP network of claim 1 , wherein said ISP sys- 
tem is a Multiprotocol Label Switching Virtual Pri- 
vate Network (MLS VPN) ISP. 

3. The ISP network of claim 1 or 2, wherein said black- 
hole router injects said second IP address in re- 
sponse to a Distributed Denial of Service (DDoS) 
attack on said VPN application. 



50 4. The ISP network of claim 1, 2 or 3, wherein said 
community value can be changed in real-time by 
said black-hole router. 



45 



55 



5. The ISP network of any preceding claim, wherein 
said ISP network utilizes dynamic routing protocols 
in combination with community-based route filtering 
to propagate the injected second IP address to said 
edge routers. 
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6. The ISP network of any preceding claim, wherein 
when said second number of edge routers directs 
VPN traffic, addressed for said first IP address, to 
said black hole router, said black hole router is 
adapted to receive such traffic as black-holed-traf- 
fic, said black-hole router adapted to analyze said 
black-holed traffic in order to determine a ratio of 
attack traffic to legitimate traffic. 

7. The ISP network of any preceding claim, further 
comprising at least one route reflector, each one of 
said route reflectors being connected to a different 
set of edge routers from said plurality of edge rout- 
ers, said route reflectors being adapted to update 
said edge routers with route instructions, such route 
instructions including said injected second address. 

8. An ISP network comprising: 

a plurality of edge routers; 
an application in direct or indirect electrical 
communication with a first one of said plurality 
of edge routers; 

said application having a first IP address such 
that VPN traffic addressed for said first IP ad- 
dress and entering said ISP network at any one 
of said plurality of edge routers, is routed to said 
application; 
a black-hole router; 

a router adapted to inject an instruction into 
said ISP network, such that select edge router 
(s) redirect VPN traffic, which is addressed to 
said first IP address, to said black-hole router. 

9. The ISP network of claim 8, wherein said injected 
instruction comprises a routing instruction having 
the same IP address as said first IP address, but 
with a higher preference than said first IP address 
and having a community value. 

10. The ISP network of claim 8 or 9, wherein said ISP 
network is a Multiprotocol Label Switching Virtual 
Private Network (MLS VPN) ISP 



14. The ISP network of any one of claims 8 to 13, 
wherein said router injects said instruction when 
said application is experiencing a DDoS attack. 

5 15. A method of managing a DDoS attack on an appli- 
cation within an ISP, said application having a first 
IP address, said method comprising: 

injecting a BGP routing instruction into said ISP 
10 when said DDoS attack is occurring; 

redirecting, at selected edge routers, VPN traf- 
fic addressed for said first I P address to a black- 
hole router; 

directing, at other edge routers, VPN traffic ad- 
's dressed for said first IP address to said appli- 
cation that is experiencing said DDoS attack. 

16. The method of claim 15, wherein said ISP network 
is a Multiprotocol Label Switching Virtual Private 

20 Network (MLS VPN) ISP. 

17. The method of claim 15 or 16, further comprising: 

receiving, at said black-hole router, said redi- 
25 rected VPN traffic; and 

determining an amount of attack traffic therein. 

18. The method of claim 15, 16or1 7, further comprising 
changing, in real-time one or more of the selected 

30 redirecting edge routers to a directing edge router. 

19. The method of any one of claims 15 to 18, wherein 
injecting said BGP routing instruction into said ISP 
is done by providing said BGP routing instruction to 

35 a route-reflector for disseminating said BGP routing 
instruction to other route reflectors within said isp 
network. 



40 



11. The ISP network of claim 8, 9 or 10, wherein said 4s 
router and said black-hole router are the same de- 
vice. 



12. The ISP network of any one of claims 8 to 1 1 , where- 
in said injected instruction is a Border Gateway Pro- 50 
tocol (BGP) routing instruction. 

13. The ISP network of any one of claims 8 to 12, 
wherein said black-hole router is adapted to receive 
redirected traffic from said select edge router(s) and ss 
to determine a ratio of attack VPN traffic to legiti- 
mate VPN traffic found in said redirected traffic. 
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